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COMBAT  PERFORMANCE 


Figure  1.  Situation  awareness  encompasses  more  than  data  about  the  environment;  it 
includes  how  those  data  are  interpreted  ayid predictions  are  made  by  the  human.  Some 
of  the  variables  influencing  these  processes  are  depicted  above.  Photo  courtesy  of  Tanya 
EUifritt,  Human  Engineering  Division ,  Armstrong  Laboratory. 

Situation  Awareness: 

The  Buzzword  of  the  ’90s 

Richard  W.  Pew 


Ot  was  my  pleasure  to  dis¬ 
cuss  situation  awareness 
(SA)  as  part  of  the  Human 
Engineering  Division,  Armstrong  Labo¬ 
ratory  Colloquium  Series  last  June. 
While  my  colleagues  and  I  have  been 


working  on  this  topic  for  a  couple  of 
years,  we  have  tried  to  maintain  a 
healthy  skepticism  about  exactly  what 
we  should  mean  by  it.  Indeed,  I  often 
wonder  even  whether  we  should  try  to 
Continued  on  page  2 
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work  with  it  as  an  overarching  concept 
at  all,  one  that  is  anything  more  than 
what  we  think  about  in  the  traditional 
assessment  of  human  performance  re¬ 
quirements  in  any  particular  system. 
Nevertheless,  it  has  replaced  workload 
as  the  buzzword  of  the  ’90s.  In  many 
analysts’  eyes,  achieving  situation 
awareness  has  become  a  design  crite¬ 
rion  along  with  more  traditional  per¬ 
formance  measures.  Organizations  are 
concerned  about  whether  it  is  an  indi¬ 
vidual  difference  variable  -  whether 
we  can  think  of  some  individuals  as 
having  more  of  it  than  others  -  some¬ 
thing  we  could  test  for.  Still  others  are 
asking  whether  it  is  something  specific 
that  can  be  trained.  However,  using 
SA  in  any  of  these  ways  requires  more 
than  an  everyday  understanding  of  the 
term.  It  requires  a  more  formal  defini¬ 
tion  and  in  this  article  I  would  like  to 
share  with  you  some  of  the  thoughts  of 
our  group  at  Bolt,  Beranek,  and 
Newman,  Inc.,  (BBN)  concerning  pos¬ 
sible  elements  of  a  definition. 

If  pressed  for  a  one-sentence  defini¬ 
tion,  I  opt  for  Endsley’s  (1988)  version: 

The  perception  of  the  elements  in  the 
environment  within  a  volume  of  time 
and  space,  the  comprehension  of 
their  meaning  and  projection  of  their 
status  in  the  near  future,  (p.  97) 

This  definition  captures  the  notion 
of  spatial  awareness  that  is  important 
to  piloting  examples,  but  also  leaves 
room  for  awareness  of  system  states 
for  non-moving  systems,  such  as  are 
produced  by  a  nuclear  power  plant 
safety  parameter  display  system.  It 
acknowledges  the  fact  that  it  is  not 
just  data  from  the  environment  that 
matter,  but  also  the  interpretation  of 
those  data,  utilizing  the  crew  mem¬ 
bers’  knowledge  and  experience  (see 
Fig.  1).  Finally,  it  includes  the  notion 
that  effective  SA  is  useful  for  anticipat¬ 
ing  what  is  likely  to  happen  in  the 
future  as  well  as  knowledge  of  the 


immediate  present. 

However,  measuring  SA  requires 
more  than  a  one-sentence  definition. 
For  me,  the  definition  must  include  the 
characterization  of  what  we  mean  by 
a  “situation.”  Then,  given  an  adequate 
definition  of  a  situation,  we  need  to 
know  what  we  must  be  aware  of  about 
situations.  My  colleagues  at  BBN  and 
I  adopt  the  following  as  a  working 
definition  of  a  situation: 

A  situation  is  a  set  of  environmental 
conditions  and  system  states  with 
which  the  participant  is  interacting 
that  can  be  characterized  uniquely  by 
its  priority  goals  and  response  options. 

This  definition  was  arrived  at,  at 
least  in  part,  because  we  were  trying  to 
figure  out  when  one  situation  ended 
and  another  began.  This  is  important 
because  a  coherent  definition  of  SA 
means  that  awareness  requirements 
must  change  when  situations  change. 
Otherwise  we  need  to  be  aware  of  all 
things  at  all  times,  and  the  concept  of 
SA  has  little  value.  Our  current  view  is 
that  in  complex  decision-making 
situations  an  individual  has  multiple 
goals  in  the  priority  stack.  At  any  one 
time,  one  or  two  of  those  goals  are 
paramount  and  those  goals,  and  the 
response  options  associated  with 
them  provide  the  basis  for  defining 
situation-awareness  requirements.  It 
is  to  this  part  of  the  definition  that  we 
turn  next. 

The  SA  requirements  are  the  essen¬ 
tial  elements  of  information  and  knowl¬ 
edge  needed  to  cope  with  each  unique 
situation.  Since  virtually  all  measure¬ 
ments  in  human  factors  are  relative, 
we  argue  that  measuring  SA  implies 
having  a  standard,  a  set  of  SA  require¬ 
ments,  if  you  will,  against  which  to 
compare  human  performance.  Such  a 
standard  must  encompass  an  abstract 
ideal ,  a  physically  realizable  ideal , 
and  a  practically  realizable  ideal  The 
abstract  ideal  includes  the  full  set  of 


information  and  knowledge  that  would 
make  a  contribution  to  accomplishing 
a  particular  goal.  This  is  an  abstract 
ideal  because  it  is  unconstrained  by 
the  design  of  the  crew  station  and  the 
information  that  is  actually  available  to 
the  crew  member.  Definition  of  the 
physically  realizable  ideal  introduces 
the  constraint  of  a  real  crew  station.  It 
is  the  information  and  knowledge  that 
a  crew  member  could  obtain,  given 
the  current  information  resources  in 
the  workplace,  that  is,  the  current  suite 
of  displays  and  controls.  It  places  no 
constraints  on  the  information  pro¬ 
cessing  capacities  and  limitations  of 
the  crew  member.  Finally,  we  think  in 
terms  of  a  practically  realizable  ideal, 
what  any  real  individual  might  be  able 
to  achieve  under  the  best  circum¬ 
stances,  taking  into  account  typical 
human  performance  capacities  and 
limitations.  It  sets  the  standard  against 
which  to  evaluate  how  well  an  indi¬ 
vidual  performed  given  the  system  he 
or  she  had  to  work  with. 

The  definition  of  the  abstract  ideal 
helps  us  to  understand  what  might  be 
accomplished  with  better  design  and 
implementation.  The  physically  real¬ 
izable  ideal  and  the  practical  ideal 
provide  a  basis  for  addressing  the 
potential  for  training  and  selection. 
They  also  might  help  us  decide  on 
format  and  layout  to  get  the  most  from 
human  performance. 

When  discussing  SA  we  make  a 
distinction  between  the  process  of 
achieving  it,  sometimes  called  situa¬ 
tion  assessment,  and  the  product  or 
awareness  that  results.  The  process  is 
defined  in  terms  of  the  demands  for 
assessing  the  relevance,  procedural 
implications,  and  urgency  of  incom¬ 
ing  data  as  well  as  initiating  goal- 
oriented  information-seeking  behav¬ 
ior.  The  process  involves  active  asso¬ 
ciation  between  knowledge  stored  in 
memory  and  currently  arriving  infor¬ 
mation,  and  it  leaves  traces  in  memory 
at  varying  levels  of  accessibility.  The 
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skilled  performer  has  learned  what 
information  to  keep  in  the  most  readily 
accessible  stores  and  how  to  change 
these  priorities  dynamically  as  a  func¬ 
tion  of  the  changing  situation. 

Endsley  (1988)  argues  that  the  prod¬ 
uct  can  be  represented  at  three  differ¬ 
ent  levels.  The  information  level  re¬ 
flects  the  literal  data  or  state  informa¬ 
tion  that  is  utilized  for  decision  mak¬ 
ing.  The  interpretation  level  is  con¬ 
cerned  with  understanding  the  impli¬ 
cations  of  the  data.  The  prediction 
level  is  concerned  with  anticipating 
likely  future  states. 

The  distinction  between  process  and 
product  is  important  because  we  might 
be  tempted  to  just  measure  the  prod¬ 
uct  when  we  are  assessing  SA;  how¬ 
ever,  improving  SA  is  more  likely  to 


involve  influencing  the  process,  either 
through  design,  selection,  or  training. 
We  had  better  find  ways  to  measure  it 
as  well. 

Perhaps  many  of  us  are  leaping 
on  the  bandwagon  prematurely.  There 
is  still  a  need  for  fundamental 
demonstrations  that  SA  may  be  re¬ 
garded  as  a  general  concept  or  capac¬ 
ity  that  can  be  differentiated  from 
operational  human  performance  it¬ 
self.  Let  me  suggest  three  ways  that 
this  could  be  accomplished: 

(1)  It  would  be  nice  to  conduct  a 
study  in  which  subjects  were  pre¬ 
tested  on  a  battery  of  tests  selected 
specifically  because  the  measured 
sensory,  perceptual,  and  cognitive 
characteristics  are  thought  to  contrib¬ 
ute  to  SA.  For  this  to  be  a  fair  test,  it 


would  also  be  necessary  to  assess 
characteristics  which,  although  equally 
important  to  overall  performance, 
were  specifically  not  considered  a 
part  of  SA.  This  latter  selection  is  the 
hard  part.  Then  select  a  practical  task 
for  which  SA  is  considered  particu¬ 
larly  important.  If  the  individual 
differences  associated  with  SA  signifi¬ 
cantly  contributed  to  performance 
variance  in  the  practical  task,  while 
the  others  did  not,  then  I  would  con¬ 
sider  such  findings  to  be  supporting 
evidence  for  the  concept  of  SA.  The 
Armstrong  Laboratory  has  been  at¬ 
tempting  at  least  the  first  half  of  this 
demonstration  over  the  past  year  us¬ 
ing  air-to-air  combat  as  the  criterion 
task.  However,  even  if  successful,  with- 
Continued  on  page  4 


TAKE  A  LOOK! 


New  Concepts  in  Human  Factors  Coming  this  Spring 


Tools  for  Automated  Knowledge  (TAKE) 
from  the  Human  Engineering  Division,  Armstrong  Laboratory 


A  concept  map  derived  from  TAKE 


■  mm  oncept  Mapping  software  for  use  during  system  specification 
hhM  and  development,  user  requirement  identification,  function 
identification,  and  task  analysis.  TAKE  is  designed  to  help  you  map, 
organize,  categorize,  and  retrieve  the  volumes  of  information  provided 
by  subject-matter  experts  and  end-users  during  knowledge  elicitation. 
TAKE  runs  on  a  Macintosh  Computer  System  7.0. 


Look  for  the  article  in  the  next  issue  of  Gateway.  TAKE  will  be 
available  this  Spring.  For  further  information ,  contact  the  CSERIAC 
Technology  Transfer  Analyst  at  (513)  255-4842. 
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COTR  Speaks 

Reuben  L.  Hann 


©ituation  awareness  has 
become  a  “hot”  area  in 
human  factors  and  ergo¬ 
nomics.  While  everyone  seems  to  be 
using  the  term,  few  have  attempted 
to  develop  a  standard  definition. 
In  our  feature  article,  Dr.  Dick  Pew 
of  BBN  offers  us  a  definition  and 
delineates  the  essential  elements  of 
situation  awareness.  In  addition,  he 
suggests  several  ways  in  which  the 
concept  of  situation  awareness  can 
be  distinguished  from  operational 
human  performance.  This  informative 
article  is  based  on  Dick’s  lecture  last 
summer  as  the  seventh  and  final 
speaker  in  the  1993  Human  Engineer¬ 
ing  Division,  Armstrong  Laboratory 
Colloquium  Series:  The  Human-Com¬ 
puter  Interface. 

Dr.  Gary  Klein  of  Klein  Associates 
was  the  fourth  speaker  in  the  1993 
Colloquium  Series,  where  he  presented 
the  latest  information  on  naturalistic 
decision  making .  Dr.  Mike  McNeese 
from  the  Design  Technology  Branch 


in  our  division,  summarizes  the 
lecture  in  this  issue.  I  had  the  chance 
to  meet  with  Dr.  Klein  for  an  extended 
interview.  An  edited  transcript  of 
my  conversation  with  Gary  follows 
Mike’s  summary. 

On  the  topic  of  naturalistic  deci¬ 
sion  making,  Klein  Associates  and 
CSERIAC  have  joined  forces  to 
present  the  Second  Conference  on 
Naturalistic  Decision  Making  at  the 
Dayton  Marriott  Hotel  June  13-15, 
1994.  A  full-page  announcement 
providing  details  of  the  conference 
appears  on  p.  10  of  this  issue  of 
Gateway. 

Intelligent  Vehicle-Highway  Systems 
(IVHS)  are  the  road  to  the  future,  quite 
literally!  The  Federal  Highway  Admin¬ 
istration  (FHWA)  has  been  hard  at 
work  to  ensure  that  human  factors  are 
considered  in  their  design.  Elizabeth 
Alicandri  of  the  FHWA’s  IVHS  Human 
Factors  Group  examines  four  of  the 
IVHS  projects  under  study. 

Triangle  Research  Collaborative 


(TRC)  has  developed  a  set  of  com¬ 
puter  tools  to  assist  in  collecting  obser¬ 
vational  data  that  reduce  the  time 
involved  and  number  of  errors.  These 
tools  are  called  the  OSC  Tools,  and 
Dave  Randle  and  Ted  Szostak  from 
TRC  have  prepared  a  report  on  them 
for  Gateway  readers. 

If  you  have  never  had  the  chance  to 
meet  any  of  our  CSERIAC  staff  mem¬ 
bers,  we  offer  you  the  next  best  thing: 
The  back  cover  of  the  1994  Human 
Factors  and  Ergonomics  Society  Direc¬ 
tory  and  Yearbook  will  have  a  photo 
of  the  whole  crew.  At  least  you  will 
be  able  to  put  a  face  with  the  voice 
the  next  time  you  have  any  contact 
with  us.  It’s  a  fine  group  of  dedicated 
people;  we  hope  you’ll  let  them  help 
by  contacting  CSERIAC  to  discuss  your 
ergonomics  problems.  # 

Reuben  “Lew”  Hann,  Ph.D.,  is  the  Con¬ 
tracting  Officer’s  Technical  Representative 
(COTR)  who  serves  as  the  Government 
Manager  for  the  CSERIAC  Program. 


Situation  Awareness  from  page  3 
out  addressing  the  second  (non-SA) 
half,  it  will  be  hard  to  argue  that  they 
have  done  any  more  than  predicted 
individual  differences  associated  with 
air-to-air  combat. 

(2)  Second,  a  similar  demonstration 
should  be  made  that  generic  training 
for  SA,  with  training  for  non-SA 
features  of  a  task  as  a  control  condi¬ 
tion,  can  make  a  differential  con¬ 
tribution  to  the  performance  of  a 
practical  task. 

(3)  Finally,  perhaps  it  comes  down 


to  the  bottom  line.  Identify  a  design 
change  that  can  produce  a  quantifi¬ 
able  improvement  in  SA  per  se.  If 
SA  is  really  important,  then  we  would 
expect  the  staff  responsible  for  design 
decisions  to  be  willing  to  invest  re¬ 
sources  to  make  that  change.  To  the 
extent  that  we  have  convinced  the 
client  that  SA  is  important,  we  should 
no  longer  have  to  prove  each  time 
that  it  makes  a  difference  in  overall 
performance. 

As  usual,  the  ultimate  test  is  in  the 
market  place.  • 


Richard  W.  Pew,  Ph.D.,  is  a  Principal 
Scientist  and  the  Manager  of  the 
Experimental  Psychology  Department  at 
BBN  Systems  and  Technologies,  Division 
of  Bolt,  Beranek,  and  Newman,  Inc., 
Cambridge,  MA. 
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Human  Factors  and  Ergonomics  Society 
Placement  Service 

Human  Factors  and  Ergonomics  Society 
(HFES)  offers  an  opportunity  for  employers 
seeking  human  factors/ergonomics  expertise  to 
tap  into  the  tremendous  knowledge  base  and 
range  of  experience  of  HFES  members  and 
other  professionals  in  related  fields. 

The  HFES  Placement  Service  can  help  in 
filling  full-time,  part-time,  or  consulting  posi¬ 
tions.  Both  members  and  nonmembers  may  use 
the  service  for  renewable  four-month  terms. 
Anonymity  will  be  maintained  if  requested  by 
the  user. 


Call  HFES  at  (310)  394-181 1  or  fax  (310)  394- 
2410  to  obtain  rates  and  application  forms. 

BCPE  Certification  Requirements 
Revised 

The  Board  of  Certification  in  Professional 
Ergonomics  (BCPE)  has  revised  its  certification 
requirements  for  ergonomists  and  human 
factors  professionals.  Starting  January  1,  1994, 
certification  criteria  are  (1)  a  master’s 
degree  in  ergonomics/human  factors  or 
equivalent  educational  background;  (2)  four 
years  of  full-time  professional  practice  in 
ergonomics  with  an  emphasis  on  ergonomic 


design;  (3)  submission  of  a  work  product 
demonstrating  the  application  of  ergonomics  to 
a  product,  process,  or  environment;  and  (4)  a 
passing  score  on  a  written  certification 
examination  administered  by  the  BCPE. 

Applications  are  available  for  $10  and  the 
application  processing  fee  is  $200  (to  which  the 
$10  application  fee  may  be  applied),  and  the 
annual  renewal  fee  is  $75.  For  further  informa¬ 
tion  or  an  application,  please  contact: 

BCPE 

Office  of  the  Executive  Director 

P.O.  Box  2811 

Bellingham,  WA  98227-2811 

Telephone  (206)  671-7601,  fax  (206)  671-7681 


Calendar 


Notices  for  the  calendar  should  be  sent  at  least  four  months  in  advance  to: 

CSERIAC  Gateway  Calendar,  AL/CFH/CSERIAC  Bldg  248,  2255  H  Street,  Wright-Patterson  AFB  OH  45433-7022 
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Human  Engineering  Division,  Armstrong  Laboratory  Colloquium  Series 

Naturalistic  Decision  Making: 

Implications  for  Design 

Gary  Klein 

Synopsis  by  Michael  D.  McNeese 


Editor’s  Note:  Following  is  a  synopsis  of  a 
presentation  by  Dr.  Gary  Klein ,  Klein 
Associates ,  as  the  fourth  speaker  in  the 
1993  Human  Engineering  Division, 
Armstrong  Laboratory  Colloquium  Series: 
The  Human-Computer  Interface.  This 
synopsis  was  prepared  by  Dr.  Michael  D. 
McNeese,  Research  Psychologist  with  the 
Design  Technology  Branch,  Human 
Engineering  Division,  Armstrong 
Laboratory.  JAL 

Oow  do  people  make  good 
decisions  under  bad  cir¬ 
cumstances?  As  Gary  Klein 
posed  this  basic-level  question,  the 
audience  was  led  to  con¬ 
sider  what  naturalistic  deci¬ 
sion  makingQADWl )  is,  what 
it  is  not,  and  how  it  can 
inform  design.  While  mak¬ 
ing  decisions  in  real-world 
situations,  people  often  en¬ 
counter  time  pressure,  ill- 
defined  goals,  changing 
conditions,  impoverished  in¬ 
formation,  learned  cues,  and 
mutual  coordination  with 
others.  These  features  were 
identified  as  representative 
of  NDM  and  have  been 
found  to  be  recognition- 
primed.  This  means  an  ex¬ 
pert  rapidly  recognizes  a 
familiar  situation,  through 
experience  and  pattern¬ 
matching  abilities,  and 
primes  a  first  option  without 
comparing  a  large  set  of  al¬ 
ternatives.  Options  are  gen¬ 
erated  one  at  a  time  until  an 
expert  finds  one  that 
satisfices.  The  satisficing  Figure  1. 


option  immediately  evokes  a  proto¬ 
typical  set  of  goals,  expectations,  and 
actions  for  use. 

In  contrast,  classical  analytical  ap¬ 
proaches  to  decision  making  (e.g., 
Multi-Attribute  Utility  Analysis)  fail  to 
answer  the  question  posed  to  the 
audience.  These  approaches  may 
work  when  several  preconditions 
are  established  (e.g.,  having  a  very 
good  knowledge  base,  reliable  data, 
and  ample  time  to  perform  analysis). 
But  typically',  real-world  situations 
fail  to  meet  such  assumptions.  Klein 
noted  that  many  decision-support  sys¬ 
tems  predicated  upon  classical  ap¬ 
proaches  result  in  user  rejection  or 


even  catastrophic  failure. 

To  assist  users  involved  in  NDM, 
designers  must  consider  new  ways  to 
allow  users  to  seek  more  information, 
re-assess  situations,  and  be  more  sen¬ 
sitive  to  cues.  When  a  situation  is 
unfamiliar  and  expectancies  are  vio¬ 
lated,  complications  may  occur.  To 
the  extent  possible,  an  interface  must 
afford  hypothesis  formation, 
storybuilding,  or  simply  must  allow  a 
user  to  understand  “what’s  happen¬ 
ing.”  In  the  words  of  Klein’s  model, 
user  assistance  could  be  provided  in 
the  form  of  facilitating  mental  simula¬ 
tion  or  progressive  deepening  of 
knowledge  intrinsic  to  the  situation  at 


A  concept  map  showing  methods  of  cognitive  task  analysis. 
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hand.  Hence,  the  evaluation  and  im¬ 
provement  of  an  option  are  enhanced 
by  an  interface. 

One  difficulty  in  design  is  ascertain¬ 
ing  “What  are  the  tough  decisions 
people  make?”  Interface  design  may 
proceed  through  cognitive  systems 
engineering ,  an  approach  that  allows 
designers  to  be  informed  by  what  we 
know  from  cognitive  science,  new 
technologies,  and  cognitive  task  analy¬ 
sis.  Cognitive  task  analysis,  an  impor¬ 
tant  component  of  the  approach,  pro¬ 
vides  a  rich  understanding  of  how 
people  make  decisions  and  in  turn 
supplies  a  means  for  evaluating  sys¬ 
tems  design.  It  is  based  on  cognitive 
considerations,  goes  deeper  yet  is  more 
selective  about  knowledge  used,  re¬ 
quires  moderate  resources,  describes 
strategies  people  actually  use,  and  is 
easy  to  communicate  but  difficult  to 
perform.  By  comparison,  traditional 
task  analysis  is  more  behavior-ori¬ 
ented,  tends  to  capture  shallow  knowl¬ 
edge,  requires  extensive  resources, 
acquires  “recipe-like”  responses,  points 
to  algorithmic  interpretations  of  hu¬ 
man  performance,  and  is  hard  to  com¬ 
municate  but  easy  to  perform.  Klein 
identified  several  tools  of  cognitive 
task  analysis  including  critical  inci¬ 
dents,  questionnaires,  controlled  ob¬ 
servation,  protocol  analysis,  repertory 
grids,  and  concept  mapping  (see  Fig. 
1).  He  mentioned  his  use  of  cognitive 
systems  engineering  on  several  ap¬ 
plied  projects  (e.g.,  a  case-based  rea¬ 
soning  system  to  support  bidding  esti¬ 
mates  and  manufacturing  plans 
for  aircraft  parts,  new  human-com¬ 
puter  interfaces  for  work  stations  on 
AEGIS-class  cruisers,  and  the  Joint 
Surveillance  Target  Attack  Radar  Sys¬ 
tem  (J STARS]). 

After  summarizing  NDM  and  cogni¬ 
tive  systems  engineering,  Klein  pre¬ 
sented  the  redesign  of  the  Airborne 
Warning  and  Control  Systems  (AWACS) 
weapon  director’s  station  as  an  ex¬ 
ample  which  reflected  implications  of 
these  approaches  for  design.  • 

Gary  Klein,  Ph.D.,  is  the  Chief  Scientist  of 
Klein  Associates,  Inc.,  Fairborn,  OH. 


Scenes  from  the  Human  Engineering  Division, 
Armstrong  Laboratory  Colloquium  Series: 


Dr.  Klein  addressed  attendees  from  Wright-Patterson  Air  Force  Base,  Wright  State  University, 
and  the  University  of  Dayton  on  the  topic  of  naturalistic  decision  making. 


Mr.  Ken  Klauer,  CSERIAC  Technical  Analyst,  asked  Dr.  Klein  for  further  information  on 
naturalistic  decision  making. 
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Human  Engineering  Division,  Armstrong  Laboratory  Colloquium  Series 

A  Conversation  with  Gary  Klein 

Reuben  L.  Hann 


Editors  Note:  Following  is  an  edited 
transcript  of  a  conversation  with  Dr.  Gary 
Klein,  Klein  Associates,  who  had  just  made 
a  presentation  as  the  fourth  speaker  in  the 
1993  Human  Engineering  Division, 
Armstrong  Laboratory  Colloquium  Series: 
The  Human-Computer  Interface.  The 
interviewer  was  Dr.  Lew  Hann,  CSERIAC 
COTR.  JAL 

OSERIAC:  How  did  you 
first  get  interested  in  the 
area  of  naturalistic  deci¬ 
sion  making? 

Dr.  Klein:  I  left  academia  to  work  at 
Wright-Patterson  Air  Force  Base.  At 
the  university  I  had  been  working  in 
the  area  of  information 
processing  and  memory, 
and  this  was  a  wonder¬ 
ful  opportunity  to  see 
how  the  research  I  had 
been  doing  related  to 
the  practical  issues  they 
were  dealing  with  at  the 
Human  Resources  Labo¬ 
ratory.  We  were  work¬ 
ing  in  developing  simu¬ 
lators  for  aircrews,  and  we  had  people 
who  were  being  asked  to  trade  actual 
flying  for  simulator  flying.  It  was  really 
a  question  of  having  them  bet  the  lives 
of  the  people  they  were  training. 

That  raised  a  lot  of  questions  such 
as,  just  what  is  expertise?  What  does  it 
mean  to  develop  it?  These  were  ques¬ 
tions  that  were  not  asked  at  that  time 
in  academia;  they  were  not  a  part  of 
the  information-processing  approach 
to  studying  behavior.  A  real  influence 
on  me  during  this  time  was  a  book  by 
Bert  Dreyfus,  What  Computers  Can't 
Do ,  which  was  a  critique  of  artificial 
intelligence.  To  me  it  was  also  a  cri¬ 
tique  of  the  information  processing 


approaches  to  try  to  break  expertise 
down  into  components,  into  context- 
free  elements  which  can  work  in  well 
controlled  tasks,  but  don’t  apply  in 
real-world  settings.  As  a  result,  I  felt 
we  needed  to  understand  what  makes 
people  proficient  and  highly  compe¬ 
tent  in  naturalistic  settings.  That  began 
my  disenchantment  with  the  tradi¬ 
tional  information-processing  frame¬ 
work  and  my  eagerness  to  explore 
other  approaches.  It  really  “de-cen- 
tered”  me.  The  classical  decision  view 
centered  around  teaching  people  to 
use  prescriptive  methods.  In  contrast, 
we  were  interested  in  decisions  people 
have  to  make  in  operational  settings, 
about  what  makes  those  decisions 
hard,  and  how  to  support  them. 


CSERIAC:  Is  there  a  “father”  of 
NDM,  or  did  it  just  evolve? 

Dr.  Klein:  It  evolved  in  the  mid¬ 
eighties.  We  had  our  first  conference 
in  1989,  when  we  had  been  working 
on  our  own  approach,  and  had  com¬ 
pleted  our  initial  studies,  then  we 
discovered  the  work  of  Jens  Rasmussen. 
Also  there  were  people  like  Marvin 
Cohen,  David  Woods,  and  David 
Noble  and  other  European  research¬ 
ers.  They  were  all  following  the 
parallel  idea  of  how  people  actually 
make  decisions — or  as  Ed  Salas  calls 
it,  decision-making  in  situ.  That  is, 
rather  than  follow  prescriptive  meth¬ 


ods,  it  was  felt  we  should  adopt  a  spirit 
of  inquiry  where  we  try  to  find  out 
what  people  really  do,  before  we  try 
to  help  them.  So,  I  think  it  started  in 
several  places  at  the  same  time.  That’s 
why  we  held  the  conference  in  ‘89,  to 
bring  those  researchers  together.  The 
expression  naturalistic  decision 
making  was  coined  at  about  the  time 
of  the  conference. 

CSERIAC:  The  notion  of  time  pres¬ 
sure  seems  to  be  an  important  part  of 
NDM’s  appeal.  The  traditional  ap¬ 
proaches  seem  to  break  down  when 
time  pressure  increases,  which  is  a  fact 
of  life  in  real-world  settings. 

Dr.  Klein:  Right.  Zakay  and  Wooler 
found  that,  although 
you  try  to  teach 
people  to  use  multi¬ 
attribute  utility  analy¬ 
sis,  as  the  time  pres¬ 
sure  increases,  you 
find  no  impact  of 
that  training.  One  in¬ 
terpretation  is  that, 
with  time  pressure, 
you  revert  to  old 
habits.  But  I  draw  a  different  conclu¬ 
sion — that  it  is  physically  impossible 
to  perform  a  rigorous  analysis  under 
time  pressure.  Interestingly,  we  saw 
the  same  naturalistic  strategies 
when  we  studied  design  engineers, 
involved  in  a  project  extending  over 
several  months.  Even  with  the  low 
time  pressure,  we  found  little  evi¬ 
dence  of  decision  analysis  or  multi¬ 
attribute  utility  analysis.  That  sur¬ 
prised  us.  We  found  they  liked  to 
know  some  of  the  options,  but  they 
used  a  lot  of  analogical  reasoning, 
where  they  would  employ  ideas  that 
worked  before.  They  took  advantage 
of  their  experience.  Where  they  had 


“Expertise  is  more  than  the  knowledge  in  the 
expert’s  head;  the  expert  sees  a  different  world 
than  others  do.” 
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no  experience,  they  would  try  to  get 
it  by  making  little  mockups  and  col¬ 
lecting  the  reactions  of  colleagues. 

CSERIAC:  Are  you  still  grappling 
with  the  concept  of  expertised 

Dr.  Klein:  For  the  past  ten  years 
or  so  researchers  have  become  inter¬ 
ested  in  what  expertise  means,  and 
how  that  affects  situational  assess¬ 
ment — the  ability  to  size  up  what 
is  happening.  Experts  do  this  well. 
Also,  we  see  that  an  important  part 
of  expertise  is  that  experts  can  see 
what  isn't  there.  They  know  what  to 
expect  and  can  detect  when  some¬ 
thing  is  missing.  David  Noble  calls 
these  negative  cues.  Novices  do  not 
have  any  expectations,  so  they  can¬ 
not  say  something  did  not  happen. 
Furthermore,  experts  can  make  dis¬ 
criminations  that  go  unnoticed  by 
novices.  A  horse-show  judge  can 
“see”  things  which  are  invisible  to 
most  of  us.  Expertise  is  more  than  the 
knowledge  in  the  expert’s  head;  the 
expert  sees  a  different  world  than 
others  do. 

CSERIAC:  Expertise  is  more  than 
just  experience,  then? 

Dr.  Klein:  Oh  yes.  In  our  studies 
of  firefighters,  we  found  that  some¬ 
one  coming  from  a  rural  area,  where 
they  had  been  exposed  to  a  limited 
number  of  fire  situations — even 
though  they  had  ten  years  experi¬ 
ence — did  not  have  the  same  level  of 
expertise  as  somebody,  say,  from 
Cleveland  with  only  a  couple  of 
years  experience,  had.  The  latter 
group  had  seen  a  wide  range  of  situ¬ 
ations.  They  knew  what  was  typical 
and  could  detect  when  something 
was  not  behaving  as  it  should. 

The  development  of  expertise  is 
extremely  important.  The  infor¬ 
mation  processing  tradition  geared 
us  to  think  that  expertise  was  just 
accumulating  more  declarative  knowl¬ 
edge — more  rules — and  increasing 
access  to  those  rules;  it  missed  the 
perceptual  aspect  of  the  process. 


The  rules  always  sound  straight¬ 
forward  when  you  find  out  what  they 
are,  but  the  difficult  part  is  being  able 
to  specify  the  antecedent  condition. 

CSERIAC:  An  area  which  has  al¬ 
ways  been  of  considerable  interest  to 
the  operational  world,  especially — 
but  not  limited  to — the  Department  of 
Defense,  is  decision  aiding.  I  assume 
principles  from  NDM  can  be  useful  in 
designing  these  tools. 

Dr.  Klein:  Very  much  so.  We  have 
been  engaged  in  this,  and  so  have 
others.  David  Noble  has  a  software 
package,  the  Recognition-Primed 
Decision  tool,  which  is  a  very 
elegant  way  of  using  feature  matching 
to  help  people  develop  situation  as¬ 
sessment  and  to  compare  different 
hypotheses  about  what  is  happening 
in  a  situation. 

CSERIAC:  Situation  assessment  is  a 
hot  area  in  itself,  and  of  course  it’s  a 
precursor  to  making  any  decision. 

Dr.  Klein:  Exactly.  From  our  find¬ 
ings,  that’s  where  the  action  is.  Usu¬ 
ally,  once  experts  know  what’s  going 
on  in  a  situation  it’s  pretty  clear  what 
to  do.  They  don’t  have  to  wrestle  with 
alternatives,  and  that’s  why  classical 
prescriptive  strategies  are  not  interest¬ 
ing.  They  are  not  relevant  to  experts. 
The  game  is  really:  What’s  going  on? 

CSERIAC:  Once  you  know  the  situ¬ 
ation,  the  answer  jumps  out  at  you. 

Dr.  Klein:  Precisely — if  you  have 
expertise.  Now,  in  terms  of  building 
decision  aids,  I  think  this  illustrates 
the  differences  between  naturalistic 
and  classical  approaches.  Classical 
approaches  were  looking  at  a  very 
tough  problem:  How  do  people 
compare  options?  They  developed 
prescriptive  strategies  for  doing 
that  as  rigorously  as  possible,  such 
as  multi-attribute  utility  analysis. 
They  created  computer-based  aids 
which  flowed  naturally  from  the 
theories.  The  idea  was  that  they 


would  be  general,  because  the  same 
issues  would  appear  time  after 
time.  This  approach  has  had  some 
success.  It  has  been  useful  in  situa¬ 
tions  where  we  don’t  have  sufficient 
expertise,  and  we  need  to  puzzle 
it  out  ourselves.  However,  to  me  it  is 
still  a  solution  searching  for  a  prob¬ 
lem.  The  naturalistic  approach 
would  be  to  identify  the  decision 
requirements  in  a  task  and  to  say 
that  these  are  going  to  drive  the 
design  of  the  system  and  the  inter¬ 
face.  We  don’t  come  in  with  a 
set  of  solutions  ready  to  implement. 
We  of  course  have  the  experience  of 
what  we  have  used  before,  but  we 
are  more  focused  on  what  the 
decision  requirements  are  for  this 
arena,  and  to  use  them  to  define  the 
specifications  for  the  system.  It 
doesn’t  completely  close  the  gap; 
there  still  is  a  requirement  for  creativ¬ 
ity  on  the  part  of  the  designer.  But  it 
gives  the  designer  the  basis  for  using 
that  creativity. 

CSERIAC:  A  final  question:  If  you 
had  all  the  resources  you  needed, 
where  within  this  area  would  you 
direct  your  effort? 

Dr.  Klein:  I  would  invest  my 
efforts  in  problem  detection.  It’s  an 
important  area.  It’s  not  how  some 
people  notice  cues  that  others  never 
even  saw.  Rather,  given  that  two 
people  have  the  same  information, 
why  does  one  person  say,  “Hmm, 
looks  like  a  problem  could  be  arising. 
I  had  better  start  preparing  contin¬ 
gencies,”  while  the  other  person 
ignores  the  information  or  explains 
it  away  until  it  becomes  such  a  dra¬ 
matic  problem  that  he  or  she  must 
attend  to  it,  and  by  then  some  of 
the  options  have  been  foreclosed 
and  the  person  is  behind  the  curve. 
It’s  a  “psychological  threshold”  ques¬ 
tion.  Why  is  this  problem  detection 
threshold  different  for  different 
people?  We  have  seen  this  phenom¬ 
enon  enough  to  know  that  this  is 
an  important  area  in  which  to  direct 
our  energies.  • 
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The  Second  Conference  on 
NATURALISTIC  DECISION  MAKING 


June  13-15, 1994 


Dayton  Marriott,  Dayton,  Ohio 


aturalistic  Decision  Making  (NDM)  research  studies  the  decision  strategies  people  actually  use  in 
bringing  their  experience  to  bear  under  challenging  real-world  conditions.  Some  of  these  conditions  are 
time  pressure,  high  stakes,  multiple  players,  and  competing  goals.  NDM  research  is  emerging  as  a  new  paradigm 
within  the  field  of  decision  research.  Its  focus  of  interest  is  on  real-world  situations,  where  both  the  context  in 
which  decisions  are  made  and  also  the  individual’s  experience  are  critical  to  diagnosing  the  situation  and 
generating  a  course  of  action. 

The  Second  Conference  on  Naturalistic  Decision  Making  will  bring  together  leading  researchers  for  panel 
sessions  which  focus  on  applying  the  findings  that  have  emerged  from  their  research.  In  addition,  empirical 
and  theoretical  papers  will  be  presented  to  challenge  and  push  forward  our  understanding  of  Naturalistic 
Decision  Making. 


The  registration  fee  includes  meals  and  is  $170  before  May  25 , 1994  and$195  after  May  25, 1994.  Registration 
is  limited  to  150.  Questions  or  comments  regarding  the  format  or  content  of  this  conference  should  be  directed 
to  Klein  Associates  Inc.  at  Voice:  (513)  873-8166,  FAX:  (513)  873-8258,  or  EMAIL:  76360.3035(a) 
COMPUSERVE.COM 

To  register  or  receive  a  brochure,  contact  Wes  Grooms  atCSERIAC,  Voice:  (513) 255-5498,  or  FAX:  (513)  255-4823 
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Federal  Highway  Administration 

Human  Factors  Program  for  Intelligent 
Vehicle-Highway  Systems 

Elizabeth  Alicandri 


Ontelligent  Vehicle-High¬ 
way  Systems  (IVHS)  are  a 
major  U.  S.  Department  of 
Transportation  initiative  to  improve 
the  safety  and  efficiency  of  the  nation’s 
highways.  The  IVHS  Program  is  a 
cooperative  effort  by  private  industry, 
academia,  and  the  government. 
Through  application  of  advanced  tech¬ 
nologies,  IVHS  will  improve  highway 
safety,  reduce  congestion,  protect  the 
environment,  and  provide  increased 
convenience  and  mobility  to  users  of 
the  surface  transportation  system. 

Various  disciplines  are  involved  in 
the  program  to  effectively  address  tech¬ 
nical,  institutional,  legal,  and  other 
issues  that  are  of  concern  in  the  devel¬ 
opment  and  implementation  of  IVHS. 
Human  factors  is  one  of  the  emphasis 
areas  requiring  early  and  continual 
attention  throughout  IVHS  devel¬ 
opment.  The  Federal  Highway 
Administration’s  (FHWA)  Office  of 
Research  and  Development  is  taking 
the  lead  in  executing  a  comprehen¬ 
sive  human  factors  program  to  pro¬ 
vide  timely  information  essential  to 
cost-effective  and  technically  sound 
IVHS  development.  IVHS  will  sub¬ 
stantially  change  the  driving  task,  and 
appropriate  and  timely  human  factors 
guidance  is  critical  to  the  success  of 
the  program. 

Although  all  IVHS  areas  are  related, 
they  have  been  divided  into  six  inter¬ 
locking  technology  areas.  TheFHWA’s 
IVHS  Human  Factors  Program  has 
ongoing  and  planned  projects  in  the 
first  four  areas  (see  Fig.  1). 

Advanced  Traveler  Information 
Systems  (ATIS)  provide  real-time  in- 
vehicle  information  to  drivers.  ATIS 


comprises  four  major  subsystems.  In- 
Vehicle  Routing  and  Navigation  Sys¬ 
tems  determine  the  best  route  be¬ 
tween  an  origin  and  a  destination  and 
provide  directional  guidance  at  choice 
points.  In-Vehicle  Motorist  Services  In¬ 
formation  Systems  provide  in-vehicle 
access  to  a  variety  of  information  that 
might  be  available  in  a  yellow  pages 
directory.  In-Vehicle  Signing  Infor¬ 
mation  Systems  solve  the  frequently 
cited  and  contradictory  problems  of 
“too  many  signs”  or  “not  enough  infor¬ 
mation”  by  providing  in-vehicle  sign¬ 
ing  adapted  to  individual  driver  and 


trip  needs .  In-  Veh  icle  Safety  Advisory 
Warning  Systems  provide  information 
regarding  fixed  or  temporary  highway 
hazards  out  of  the  driver’s  line  of  sight. 

Advanced  Vehicle  Control  Sys¬ 
tems  (AVCS)  aid  drivers  in  controlling 
their  vehicles,  particularly  in  emer¬ 
gency  situations,  and  ultimately  will 
take  over  some  or  all  of  the  driving 
tasks.  AVCS  is  made  up  of  three 
subsystems.  Sensory  Enhancement 
Systems  use  specialized  sensors  to  en¬ 
hance  the  human  visual  system  and 
provide  an  extended  line  of  sight  in 
Continued  on  page  12 


Figure  1 .  The  technical  areas  currently  understudy  by  the  Federal  Highway  Administration 
include  Advanced  Traveler  Infoimation  Systems  (ATIS),  Advanced  Vehicle  Control  Systems 
(AVCS),  Commercial  Vehicle  Operations  (CVO),  and  Advanced  Traffic  Management 
Systems  (ATMS). 
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reduced  visibility  conditions.  Obstacle 
Detection  and  Avoidance  Systems  pro¬ 
vide  warnings  to  drivers  when  they  are 
in  danger  of  colliding  with  a  fixed  or 
moving  object,  and  will  eventually 
take  control  of  the  vehicles  in  situa¬ 
tions  where  driver  response  times  are 
insufficient  to  avoid  collision.  Auto¬ 
mated  Control  Systems  take  over  a 
portion  of  the  driving  task.  Eventually 
these  systems  will  achieve  the  sophis¬ 
tication  to  become  a  fully  automated 
highway  system  (AHS)  allowing  for 
“hands  and  feet  off”  driving. 

Commercial  Vehicle  Operations 
(CVO)  refer  to  application  of  IVHS 
technologies  to  the  special  needs  of 
commercial  vehicles  and  include  auto¬ 
mated  vehicle  identification  and  loca¬ 
tion,  weigh-in-motion,  clearance  sens¬ 
ing,  and  record  keeping. 

Advanced  Traffic  Management 
Systems  (ATMS)  monitor,  control,  and 
manage  traffic  on  streets  and  high¬ 
ways  to  reduce  congestion  using  ve¬ 
hicle  route  diversion,  automated  sig¬ 
nal  timing,  changeable  message  signs, 
and  priority  control  systems. 

Advanced  Rural  Transportation 
Systems  (ARTS)  apply  IVHS  technolo¬ 
gies  to  the  special  needs  of  rural  sys¬ 
tems  and  include  emergency  signal¬ 
ing,  vehicle  location,  and  traveler  in¬ 
formation. 

Advanced  Public  Transportation 
Systems  (APTS)  use  IVHS  technolo¬ 
gies  to  enhance  the  effectiveness, 
availability,  attractiveness,  and  eco¬ 
nomics  of  public  transportation  and 
include  fleet  management,  automated 
fare  collection  and  real-time  informa¬ 
tion  systems. 

The  products  of  the  FHWA  IVHS 
Human  Factors  Program  will  aid  the 
IVHS  community  in  planning,  design¬ 
ing,  and  implementing  an  IVHS  that 
will  meet  the  operational  objectives 
and  produce  the  anticipated  benefits. 
Ongoing  projects  in  ATIS,  AVCS,  CVO, 
and  ATMS  are  focusing  on  identifica¬ 
tion  and  resolution  of  critical  human 
factors  issues  in  these  areas.  Each  of 
these  projects  incorporates  both  ana¬ 
lytical  and  empirical  research  to  en¬ 
sure  that  all  critical  areas  are  covered 


and  that  sound  human  factors  guide¬ 
lines  and  handbooks  are  developed 
for  use  by  IVHS  designers. 

The  ATIS  program  includes  labora¬ 
tory  and  field  efforts  directed  at  iden¬ 
tifying  user  needs  and  ensuring  that 
system  design  accommodates  these 
needs.  Critical  issues  being  addressed 
include  display  modality  (visual  vs. 
auditory),  display  location  (in-dash  vs. 
head-up  [HUD]),  and  information  flow. 
The  TRAVTEK  project  in  Orlando, 
Florida,  provided  important  real-world 
information  on  ATIS,  and  issues  raised 
during  that  operational  test  will  be 
further  researched  in  controlled  labo¬ 
ratory  and  simulation  environments. 

The  AVCS  program  focuses  on  the 
development  needs  of  the  automated 
highway  system  (AHS).  FHWA  has 
been  congressionally  mandated  to 
demonstrate  the  AHS  by  1997,  and 
human  factors  considerations  are  be¬ 
ing  given  considerable  weight  in  sys¬ 
tem  design.  Critical  issues  including 
drivers’  abilities  to  enter,  exit,  and 
maneuver  in  an  automated  system  are 
being  investigated  using  the  Iowa  Driv¬ 
ing  Simulator  as  the  primary  test  bed. 
Further  investigations  will  address  user 
acceptance  of  the  short  headways  en¬ 
visioned  for  AHS,  methods  of  transfer¬ 
ring  control,  and  the  effects  of  reduced 
capability  AHS  on  driver  performance. 

The  focus  in  the  CVO  program  is 
similar  to  the  ATIS  program,  but  ad¬ 
dresses  a  different  user  population, 
different  vehicle  characteristics,  and 
different  information  needs.  The  CVO 
human  factors  project  examines  the 
information  requirements  that  are  nec¬ 
essary  for  the  successful  operation  of 
the  trucking  system. 

In  the  ATMS  program,  human  fac¬ 
tors  issues  associated  with  the  design 
and  operation  of  traffic  management 
centers  (TMC)  are  of  primary  impor¬ 
tance.  Specific  human  factors  ques¬ 
tions  addressed  in  this  program  in¬ 
clude  investigation  of  the  number  and 
type  of  staffing  required  for  a  TMC 
under  both  normal  and  emergency 
situations  and  an  analysis  of  the  ergo¬ 
nomics  of  the  TMC  to  optimize  infor¬ 
mation  throughput  and  user  require¬ 


ments.  Another  key  research  area 
addresses  the  type  of  decision  aids 
required  for  efficient  TMC  operations. 
Much  of  this  work  will  be  performed 
on  a  human  factors  research  TMC 
simulator. 

IVHS  has  great  potential  to  signifi¬ 
cantly  increase  highway  safety  and 
decrease  congestion,  if  it  is  designed 
with  appropriate  consideration  for 
the  needs  and  capability  of  the  di¬ 
verse  driving  population.  The  FHWA 
IVHS  Human  Factors  Program  will 
ensure  that  system  designers  have  ac¬ 
cess  to  this  critical  information  in 
both  the  initial  design  phases  and 
throughout  the  evolution  of  the  sys¬ 
tem  development.  • 

Elizabeth  Alicandri  is  the  Manager  of  the 
Human  Factors  Laboratory,  Federal  High¬ 
way  Administration  Turner-Fairbanks 
Highway  Research  Center,  McLean,  VA. 


Request  for  Topics 

For 

State-of-the-Art  Reports  (SOARS) 

CSERIAC  makes  every  effort  to  be 
sensitive  to  the  needs  of  its  users. 
Therefore,  we  are  asking  you  to  sug¬ 
gest  possible  topics  for  future  SOARS 
that  would  be  of  value  to  the  Human 
Factors/Ergonomics  community.  Pre¬ 
vious  SOARs  have  included  Hypertext: 
Prospects  and  Problems  for  Crew  Sys¬ 
tem  Design  by  Robert  J.  Glushko,  and 
Three  Dimensional  Displa  ys:  Percep¬ 
tion ,  Implication,  Applications  by 
Christopher  D.  Wickens,  Steven  Todd, 
&  Karen  Scidler.  Your  input  would  be 
greatly  appreciated.  We  are  also  look¬ 
ing  for  sponsors  of  future  SOARs. 
CSERIAC  is  a  contractually  conven  ient, 
cost  effective  means  to  produce  rapid 
authoritative  reports. 

Send  your  suggestions  and  other 
replies  to: 

CSERIAC  Program  Office 
AE/CFH/CSERIAC  Bldg  248 
ATEN:  Dr.  Ron  Schopper, 

Chief  Scientist 
2255  H  Street 

Wright-Patterson  AFB,  OH  45433-7022 
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Efficient  Event  Analysis  Through 
Multimedia  Integration 
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raditional  observational 
research  methods  are  in¬ 
herently  time-consuming 
and  error-prone.  A  new  technology 
by  Triangle  Research  Collaborative, 
Inc.  (TRC),  the  OCS  TOOLS,  inte¬ 
grates  observational  research  methods 
and  computer  and  video  technology 
into  a  single  multimedia  environment. 
This  environment  consists  of  com¬ 
puter  and  audio/video  equipment 
and  the  OCS  TOOLS  which  are  opti¬ 
mized  for  the  study  of  live  or  recorded 
events.  The  system  offers  increased 
efficiency  through  time-savings,  im¬ 
proved  accuracy,  and  flexibility  for  a 
wide  range  of  applications  requiring 
event  analysis. 

Time-Savings  Plus  Accuracy 

The  OCS  TOOLS  software  auto¬ 
matically  synchronizes  codes  with 
time  from  the  computer’s  internal 
clock,  or  to  machine-readable  time- 
code  recorded  on  audio  or  videotape. 
This  orders  codes  chronologically 
and  provides  data  for  a  number  of 
analyses.  Since  time  does  not  have 
to  be  entered  manually,  as  it  would 
using  a  stopwatch  and  data  collec¬ 
tion  sheet,  observers  are  free  to  focus 
their  attention  on  the  event  coding 
task.  In  addition  to  saving  time,  data 
collected  with  the  system  are  accurate 
to  real-time. 

Data  are  stored  to  disk  during 
data  collection  in  a  virtual  mode 
and  are  immediately  ready  for  analy¬ 
sis.  Data  transcription  is  also  elimi¬ 


nated;  again,  time  is  saved  and  accu¬ 
racy  is  increased. 

Flexibility 

Hardware  modularity,  software 
configurability,  and  dataset  adaptabil¬ 
ity  increase  functionality  and,  there¬ 
fore,  efficiency. 

Hardware  Modularity 
The  OCS  TOOLS  may  be  integrated 
with  a  number  of  optional  multi- 
media  modules  to  fit  a  specific  appli¬ 
cation.  Modules  include  but  are  not 


a.  Portable  workstation 


b.  Workstation  with  timecode  and 
VCR  control 


i  o 


c.  Windows  workstation  for 
two  operators 


limited  to  a  Timecode  Reader/ 
Generator  (Figure  la-d),  VCR- 
Controller  (Figure  lb-d),  Dual  Key¬ 
board  or  Keystroke  Capture  System 
(Figure  lc  and  d),  and  Video  Overlay 
or  Video  Windowing  System 
(Figure  lc  and  d).  Modules  may  be 
integrated  to  add  functionality  and 
increase  usability  according  to 
project  requirements. 

Software  Configurability 
The  OCS  TOOLS  may  be  config¬ 
ured  to  code  events  at  various  speeds 
Continued  on  page  14 


ADAPTABILITY 
FOR  MULTIPLE 
APPLICATIONS 


d.  Software  usability  and  test 
workstation 


Figure  1 .  A  range  of possible  configurations from  portable  to  fully  integrated  workstations. 
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and  degrees  of  synchronization  with 
audio  or  videotape.  Events  may  be 
coded  live  with  no  synchronization, 
at  a  freeze-frame  rate  with  or  with¬ 
out  synchronization,  and  at  any 
speed  using  machine-readable 
timecode  with  full  synchronization. 
The  tools  offer  a  variety  of  analyses 
including  frequency  and  duration 
analysis,  interval  analysis,  pattern 
analysis,  time  series  comparison,  and 
dataset  reliability.  In  addition,  data¬ 
sets  and  reports  can  be  output  in 
ASCII  format  for  use  with  other  statis¬ 
tical  packages.  For  group  projects, 
managers  may  assign  access  rights. 
The  software  offers  a  number  of  cod¬ 
ing  options,  variety  of  analyses,  and 
project  management  features.. 

Dataset  Adaptability 

Another  advantage  of  the  OCS 
TOOLS  is  their  adaptable,  generic 
method  of  defining  data  and  analyses. 
The  observer  enters  discreet  event 
codes  and  may  enter  additional  com¬ 
ments  for  each  record.  Variable  files 
are  used  to  define  the  time-ordered 
data  and  are  composed  of  discreet 
or  durational  events  (paired  discreet 
event  codes).  Variables  may  be  fur¬ 
ther  defined  using  wildcard  charac¬ 
ters  (?,*)  to  group  events.  Variable 
files  allow  the  redefinition  of  datasets 
for  many  different  analyses  without 
changing  the  original  data.  In  addi¬ 
tion,  potential  dataset  errors,  accord¬ 
ing  to  variable  definitions,  are  auto¬ 
matically  located.  This  allows  re¬ 
searchers  to  make  necessary  correc¬ 
tions  before  analysis. 

Applicability 

The  OCS  TOOLS  are  being  used  in  a 
number  of  fields.  The  same  tool-set  can 
be  used  by  human  factors  engineers, 
industrial  engineers,  ergonomists,  physi¬ 
cal  therapists,  psychiatrists,  research 
psychologists,  and  other  professionals 
to  accurately  and  efficiently  record  and 
analyze  events.  Some  existing  applica¬ 
tions  are  shown  in  Table  1. 


Table  1. 

Applications  of  the  OCS  Tools 


Technology  Assessment 

Product  testing,  software  usability,  cash  register  usability,  and  medical 
technology  assessment 

Work  Measurement  and  Task  Analysis 

Crew  performance  in  loading  and  firing  howitzers,  assembly  line  measure¬ 
ments,  incentive  pay  measurements,  and  pilot  workload 

Communications  Research 

Communication  between  multiple  helicopter  pilots  and  control  tower  per¬ 
sonnel,  management  group  communication,  marital  and  mother-child  com¬ 
munication  and  interaction,  television  content  analysis,  focus  group  re¬ 
sponse  measurement,  and  emergency  room  communication 

Medical  Studies 

Pain  studies,  carpel  tunnel  measurements,  range  of  movement  and  rehabili¬ 
tation  in  geriatrics  and  sports  medicine,  and  fetal  movement  studies 

Other  Applications 

Sociological  interaction,  traffic  studies,  facial  action  measurement,  percep¬ 
tion,  cognitive  development  in  children,  psychology  and  nursing  education, 
and  animal  studies 


Time  and  accuracy  are  important  in 
observational  research.  The  OCS 
TOOLS  integration  greatly  improves 
efficiency  and  can  be  used  in  virtually 
any  application  requiring  observational 
research. 

The  OCS  TOOLS  are  a  subset  of 
TRC’s  Multimedia  Research  Tools 
(MRT).  Other  components  include 
the  Motion  Analysis  System  Toolset 
(MAS  TOOLS™),  future  Physiological 
Data  Acquisition  Toolset  (PDAS 
TOOLS™),  and  toolsets  to  be  an¬ 
nounced  in  the  future.  For  additional 
information  on  these  tools,  contact: 

Triangle  Research  Collaborative  Inc. 
PO  Box  12167 
100  Park  Suite  115 
Research  Triangle  Park  NC  27709 
(919)  549-9093  or  (800)  467-9093 

/.  David  Randle  and  Thaddeus  K.  Szostak 
are  respectively  Technical  Writer  and 
President  of  Triangle  Research  Collabora¬ 
tive ,  Inc. 


MAILING  ADDRESS 

To  maintain  Gateway  as  a  free 
publication,  it  is  necessary  for 
us  to  keep  the  costs  down.  You 
can  help  us  do  that  by  making 
sure  we  have  your  correct  ad¬ 
dress  and  notifying  us  of  dupli¬ 
cate  mailings.  Also,  if  you  know 
of  anyone  who  would  like  to  be 
added  to  our  mailing  list,  please 
have  them  contact  us. 

Please  note  our  mailing  ad¬ 
dress. 

CSERIAC  Program  Office 
AL/CFH/CSERJLAC  Bldg  248 
ATTN:  Jeffrey  A.  Landis, 
Gateway  Editor 
2255  H  Street 
Wright  Patterson  AEB  OH 
45433-70 22 
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COLOR  IN  ELECTRONIC 
DISPLAYS 


COLOR  IN 
ELECTRONIC 
\  DISPLAYS 

EDITED  BY 

EINO  WIDDEL 

AND 

DAVID  L.  POST 


Color  in  Electronic  Displays  edited  by 
Heino  Widdel  apd  David  L.  Post  ( 1992). 
Published  by  Plenum  Press. 


olor  in  Electronic  Displays  is  a  comprehensive  collection  of 
information  concerning  color  vision,  color  perception, 
colorimetry,  and  color  displays  relevant  to  display  design.  It 
comprises  1 1  chapters,  which  are  divided  into  four  sections  devoted  to  the 
perception  and  measurement  of  color,  theoretical  and  applied  color 
research  methodologies,  use  of  color  in  electronic  displays,  and  electronic 
color-display  technology.  Leading  international  authors  contributed  to 
this  work. 


Through  special  arrangement  with  the  publisher ,  Plenum  Press , 
CSERIAC  price:  $45.  To  order  or  obtain  further  information  on 
Color  in  Electronic  Displays ,  contact  the  CSERIAC  Program  Office  at 
(513)  255-4842. 


SCOPE  IT  OUT! 


Smart  Contract  Preparation  Environment  from  the  U.S.  Army  Research  Laboratory 


SCOPE  ( Smart  Contract  Preparation  Environment) 


wo  software  programs  intended  to  assist  in  the  preparation  of 
the  request  for  proposal  (RFP): 

SPEC  Maker  -  to  facilitate  the  tailoring  of  two  commonly  used  HFE 
design  standards  (MIL-STD-  1472D,  Human  Engineering  Design  Criteria 
for  Military  Systems,  Equipment,  and  Facilities,  and  MIL-STD- 1474C, 
Noise  Limits  for  Military  Materiel)  for  incorporation  in  the  system 
specifications. 

CDRL  Maker  -  to  facilitate  the  preparation  of  DD  Forms  1423,  the 
Contract  Data  Requirements  List,  and  the  tailoring  of  the  human 
engineering  Data  Item  Descriptions  (DIDs),  DD  Forms  1664. 

Price:  $35  each.  For  further  information  on  SCOPE  and  its  two 
products,  SPEC  Maker  and  CDRL  Maker,  contact  the  CSERIAC 
Technology  Transfer  Analyst  at  (513)  255-4842. 
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CSERIAC 
PRODUCTS  AND 
SERVICES 

CSERIAC's  objective  is  to  acquire, 
analyze,  and  disseminate  timely  infor¬ 
mation  on  crew  system  ergonortiics 
(CSE).  The  domain  of  CSE  includes 
scientific  and  technical  knowledge  and 
data  concerning  human  characteris¬ 
tics,  abilities,  limitations,  physiological 
needs,  performance,  body  dimensions, 
biomechanical  dynamics,  strength,  and 
tolerances.  It  also  encompasses  engi¬ 
neering  and  design  data  concerning 
equipment  intended  to  be  used,  oper¬ 
ated,  or  controlled  by  crew  members. 

CSERIAC's  principal  products  and 
services  include: 

■  technical  advice  and  assistance; 


■  customized  responses  to  biblio¬ 
graphic  inquiries; 

■  written  reviews  and  analyses  in 
the  form  of  state-of-the-art  reports  and 
technology  assessments; 

■  reference  resources  such  as  hand¬ 
books  and  data  books. 

Within  its  established  scope,  CSERIAC 
also: 

■  organizes  and  conducts  work¬ 
shops,  conferences,  symposia,  and 
short  courses; 

■  manages  the  transfer  of  techno¬ 
logical  products  between  developers 
and  users; 

■  performs  special  studies  or  tasks. 

Services  are  provided  on  a  cost- 
recovery  basis.  An  initial  inquiry  to 
determine  available  data  can  be  ac¬ 
commodated  at  no  charge.  Special 
tasks  require  approval  by  the  Govern¬ 
ment  Technical  Manager. 


To  obtain  further  information  or  re¬ 
quest  services,  contact: 

CSERIAC  Program  Office 
AL/CFH/CSERIAC  Bldg  248 
2255  H  Street 

Wright-Patterson  AFB  OH  45433-7022 

Telephone . (513)  255-4842 

DSN . 785-4842 

Facsimile . (513)  255-4823 

Government 

Technical  Manager . (513)  255-8821 

Director:  Mr.  Don  A.  Dreesbach; 
Government  Technical  Manager:  Dr. 
Reuben  L.  Hann;  Associate  Govern¬ 
ment  Technical  Manager:  Ms.  Tanya 
Ellifritt;  Government  Technical  Director: 
Dr.  Kenneth  R.  Boff. 


CSERIAC  Gateway  is  published  and 
distributed  free  of  charge  by  the  Crew 
System  Ergonomics  Information  Analysis 
Center  (CSERIAC).  Editor:  Jeffrey  A.  Landis; 
Copy  Editor:  R.  Anita  Cochran;  Illustrators: 
Ronald  T.  Acklin,  Timothy  J.  Span;  Layout 
Artist:  Vicky  L.  Chambers;  Ad  Designer: 
Kristen  Cheevers. 
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